Note on Matrix Factorization

A large portion of Linear algebra (scientific computing) is devoted to accelerate computationally intensive operations through the deeper understanding on the structure of the matrix. While it is not extremely hard to obtain an analytical solutions for the problems of interest, often a large-size matrix or repetetive nature of the computations prohibit us from getting the actual solution in numbers. Most of the solutions contain determinants or inverse matrices and the best algorithms to compute these values or matrices without considering the special structure of the given matrix is often are too naive to deliver the solutions (real number, not symbols!) in time. Hence, we explore the ideas of scientific computing where often the matrix structure or matrices in factorized forms let us save huge amount of computational power by adding very little extra initial processing which results in order of hundreds of speed-up. Some tips that author needs to keep in mind all the time are written down in this note and hope it would be helpful to other people as well. 1 LU decomposition Objective : LU decomposition is devoted to easily solve a linear equation even without explicitly obtaining an inverse matrix. Let’s try to solve a system Ax = b where the matrix A is invertible. If A is invertible, it is proven that there is a unique factorization : A = LU where L is a unit lower triangular matrix (all diagonals are ones) and U is the upper triangular matrix. In fact, LU decomposition is exactly what Gaussian elimination is about. Now we can define the problem again as to solve LUx = b. Then, we define y ∆ = Ux. Now the problem we are trying to solve is : Ly = b Above, L is a unit lower triangular matrix. Hence, it is very straightforward to obtain the solution y (use forward substitution). Then, once y is obtained, we seek x :